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ABSTRACT 
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Introduction 



In the education profession, the concept of "account- 
abiiity” remains very much on the scene. Since the beginning 
of the seventies, many techniques have been introduced into 
pubiic education and teacher training institutions. Severai of 
these techniques have met with varying degrees of success. 
The trend toward accountabiiity appears to be of practicai use 
to educators, but it is not without iimitations and pitfaiis. The 
greatest of these appears to be the reiuctance of educators to 
make some form of commitment to a set of objective criteria to 
accompiish in an instructional program. This emphasis on 
accountability is many times misinterpreted as a fact-finding 
process, rather than an attempt to find out how effective our 
programs are. In most instances, the common question arises, 
"How do educators introduce accountability into instructional 
programs where it did not exist in the past?” 

As a part of the May, 1971, Annual Accountability Institute 
conducted by EPIC Diversified Systems Corporation in 
Tucson, Arizona, several educators who are implementing 
some of the accountability techniques were invited to present 
their activities, successes, and pitfalls. This booklet includes 
the presentations of five of these educators in the areas of: 

1. Performance Contracting 

2. Individualized Performance-Based Teacher Education 
Programs 

3. Educational Management Systems 

Their presentations are based on actual programs imple- 
mented at their respective levels. They describe the highlights 
o' their accomplishments and their pitfalls. The educators also 
offer possible solutions to these particular areas of account- 
ability. 
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The 

Educational 

Management 

System 



DR. GEORGE L. CALDWELL, SUPERINTENDENT 
SAN BERNARDINO CITY UNIFIED SCHOOL DISTRICT 
SAN BERNARDINO, CALIFORNIA 

I am a new superintendent. I have only been on the job in 
San Bernardino since July 1, 1970. Our educational manage- 
ment system, as we call it in San Bernardino, literally has been 
off the runway only eight months. I do have some concerns 
coming to you and talking about an educational management 
system in a hrge urban school district. If you have an interest 
in what we are doing, write to us and we will send you all the 
paperwork we have. We would appreciate back from you any 
ideas or criticism you have of the approach we are taking. 

The reason for going into an educational management 
system was prompted primarily by my personal commitment 
to systems approach to education. I literally ran for superin- 
tendent in the district on the basis that we had to make some 
changes in education A systematic approach to instruction 
was, if not a proven means, at least a means that had been 
applied to any great extent in the past and something that I 
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thought was worthwile. Of course, I was not dealing in a 
vacuum on tfiis. Some of you have had the exposure already 
to PPBS and know what the California system of PPBS is. 
You may or may not know that we have a Joint Legislative 
Committee on Educational Goals which has been operating 
in California. We also have a new state superintendent who 
ran on the platform of accountability. I am eager to see what 
he is going to come cut with in terms implementing this, 
because he spoke for accountability; but also against stan- 
dardized achievement tests or tests of this type. We were 
deeply involved already in a systematic approach to education 
and the writing of objectives in our ESEA Title I programs, 
ESEA Title VII programs, and Vocational Education Act pro- 
grams. In California, we are required now to use a particular 
format in our programs for mentally gifted minors. So, we had 
those external kinds of things there; but really taking on this 
task, as far as I was concerned, was one of a personal commit- 
ment. I did not play "by the book’’ because the book says when 
you are a new superintendent, you go out in the bushes for a 
year and watch what goes on and get the lay of the land and 
then come back in and get off the ground. We started right 
from the beginning in July, 1970, to design an educational 
management system. We determined that it had to be a 
humanistic type of system. 

As a school district, we are very actively opposing the 
PPBS California-type program because we do not think that 
its thrust is in the right direction. My background before 
coming to the superintendency was in business services. My 
colleagues in that area do not speak to me any more because 
my protest against PPBS in California is that it is basically an 
accounting-oriented system. I firmly believe that if there is 
going to be any value in this particular approach it has to be 
instructionally-oriented rather than business or accounting- 
oriented. So, consequently, we buried the business end of it, 
buried the budgeting end of it. We have tried to approach it 
from a point of view of instruction being the Important thing 
and that the program budgeting, which as you know has been 
mandated now in California, will grow from that. I had been 
in the business end of the game long enough to know that 
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anyone can sandbag in program budgeting. I know that it is 
virtually impossible to come up with a cost effectiveness pro- 
gram that has any great meaning to it. I think we may come to 
this in the future, but I have this strong belief at the present 
time that program budgeting as it is being designed in Cali- 
fornia is not going to be able to apply the dollar to what is 
happening to the youngsters in the classroom. 

We designed the program during the summer months. 
We did this with a great deal of assistance from a man who 
used to be associated with EPIC here in Tucson who is now 
assistant in the area of research and development in the 
school district. We went to our staff in September at a super- 
intendent’s meeting just before school started and we threw 
out our expectancies in this particular area. The expectancies 
were that every teacher in the school district would become 
involved in a systematic approach to instruction. 

We used really a three-pronged approach in discussing 
this with the teachers. We said that, in the first place, you 
know there is such a thing as PPBS and the legislature is very 
eager to get this thing off the ground. You know that there is a 
joint legislative committee and that the legislators are going 
to mandate some kind of a goal system also in California. We 
took a hard run at whether teachers wanted to believe it or hot. 
The public image of education was bad not only in California 
but nationally. We said if they did not believe that, they ought 
to start ringing doorbells up and down their street and asking 
the question. We also pointed out in that area that we did not 
believe that the man next door or the man in the legislature 
was going to be the one who was going to stand up and be 
counted for education, but it had to happen within our own 
ranks. That we, as teachers, administrators, and school board 
members, had to start something about improving the image 
of education in the eyes of the public. If that happened, then 
maybe some of these other problems fhat we have would start 
solving themselves. Then, of course, we used the third 
approach (and the only worthwhile one as far as I am con- 
cerned) that we definitely believed there is a value to indi- 
vidual teachers approaching instruction in this manner. No- 
body is going to turn around the public image of education 
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other than people in education themselves. Therefore, we 
must get something going which we can go back to the public 
and say, "look, here is what we can do and here is the 
evidence that we are doing it." 

We designed our Educational Management System to be 
a total management system; one of our hangups with PPBS 
was that we felt it was only a piece of a management system. 
We defined lines of responsibility. I know these are very 
cloudy in many, many school districts and we had this cloudi- 
ness in my school district. Prior to the Educational Manage- 
ment System, people were making the round trying to find out 
what they needed to know ahead of time — which office to 
go to to get the right answer or playing one person against 
the other. We tried to clean the whole thing up and said that 
really there were only four important stops along the way in an 
educational system. They are the student, the teacher, the 
principal, and the superintendent and that everybody else just 
existed for the purpose of providing support services to get 
things happening to kids in the classroom. So, in one action, 
we just told the principals they had only one boss and that 
boss happened to be the principal; and we said to the 
teachers, "your responsibility is really in the classroom with 
students." 

The other portion that we tried to very much emphasize 
was development. I keep saying I have a neon sign that flashes 
on and off and says this was entirely developmental. We did 
not come in and drop any set system on top of anybody. There 
is no one system. There is a process that takes place here — 
that process is finding out where you are, where you want to 
go, and making a decision about how you want to get there. 
Then having some places along the line where you stop and 
see whether in fact you are still on that same trail. 

We took all the fancy labels off the systems approach to 
education and said to teachers, "as iong as you have these 
ingredients in there, think up your own labels." This was 
specifically instruction-oriented and we talked about perform- 
ance of students as far as teachers were concerned. Now, in 
the system, as I have pointed out, I have responsibility for 
principals, so every principal in the district did write objec- 

( 
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tives in terms of his relationship to teachers, not to students. 
The principal’s performance objectives are what is going to 
happen to teachers, how he is being a resource or how he 
expects performance or product from teachers. Teachers’ 
objectives or departmental objectives in secondary schools 
are stated in terms of student performance. 

In our overall structure, we have tried to design a system 
that does take in the community goals and board goals, as 
well as district objectives, school objectives, individual teach- 
er objectives, and student objectives. We will have some 
people next year who have gone through the student objec- 
tives and are going to send the students home with their 
objectives to ask their parents for their commitment to work 
with the student. We think that is one way of completing the 
loop, because if the parents get involved in stating objectives 
to work with the students, that is going to get back to the 
Board of Education ultimately and we will have completed the 
loop in that particular area. Again, let me say that we are not 
talking about a year or two-year program. Some of the most 
optimistic people in our district think we will be there in five 
years. I keep thinking maybe it will be eight or ten years 
before we finally get there. 

Another hangup I have with some of the powers is that 
they talk about PPBS happening overnight. It is certainly not 
that simple a process. I think this whole thing has to come 
from within. There has to be a value seen by the teacher in 
participating in this particular kind of a program. 

You can go out and mandate these things; I get a lot of 
pressure right now from our Board of Education and other 
people in our community to go out and mandate human rela- 
tions programs. I resist and say that it is not going to do any 
nood to go out and mandate programs, and show the films, 
and bring in the guest speakers if there is not a commitment 
and awareness on the part of the teacher that something 
needs to happen in this particular area. So we are trying to 
battle it, at least from the point of view that there is a value 
in many of these things that can be shown to people — and 
once that value is shown, you are going to get most of them 
moving. Again, I am not sc naive as to recognize that we are 
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going to get 100%. I do not think we will ever get 100% of our 
people in San Bernardino writing objectives. In no way do I 
think we will even get much more than probably six out of ten 
writing even worthwhile objectives. 

Overall structure then does include, as I have mentioned, 
the umbrella of district-wide goals. In California, we are going 
to have above that the bigger umbrella of some State-wide 
goals. Our Board of Education had never officially gone 
through the process of deciding what are the goals of educa- 
tion in San Bernardino. There was a statement of philosophy 
and some off-shoots about things that ought to be happening 
to students, but we felt that rather than starting at the top and 
stating some district-wide goals and then building under the 
umbrella to district-wide objectives and then going to schools 
and then going to teachers — which was the direction that 
PPBS took in California — we would start down where the 
action is and ask that teachers develop objectives. We asked 
teachers to lay out their work plan, state an objective in an 
acceptable format, develop a monitoring plan, and have a 
performance review with their principal. Some of the teachers 
did a fantastic job and some others you wish you could put 
at the bottom of the waste basket. We were not the least bit 
critical. We said, okay, great. You have stated an objective, a 
work plan, let’s go through it. 

In our district, approximately 95% of our elementary 
teachers elected to participate in this voluntarily; 100% of 
our principals. One hundred percent of the people in the 
central office wrote objectives maybe because the superin- 
tendent still has some control over these people. I am amazed 
that even 50% of the senior high teachers participated. We 
set a target date of November 1, 1970, and the cries are still 
being heard around the world about getting that done between 
the first of September and the first of November. But 1 took 
a pretty hard position on this. 1 said, if we are really going to 
develop this, let’s do it. 

Then, in November, 1970, we went back to the Board 
again, saying that 1 did not want this to be a top-down kind of 
thing; asked if it was not about time the Board of Education 
started talking about what the goals of this community and 
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Board’s goals of education are. We had, for the first time in 
my life, heard some discussion around the Board table about 
such things as reading and arithmetic and social studies and 
attitudes of students and human relations and things of this 
type. They had some very, very healthy discussions — some 
arguments back and forth — and then set an interim state- 
ment of goals for the district. We sent the goals out to all the 
business men in the district, PTA groups, to a sampling of 
parents in the district, to people in the district, to all teachers 
and so on, and we have been having meetings and discussions 
and getting communications back as to whether these goals 
are acceptable or not. 

What we intend to do under the goal statements once 
they are adopted (and we expect the Board to adopt them in 
June) will be to design district-wide objectives stated in a 
format that is basically very similar to what you have been 
hearing here at EPIC. We found out in a hurry when we started 
talking about district-wide objectives that you do not get away 
with this business of 60% of the students achieving. You do 
not get by with 80% or 90%, because the minute you have a 
percentage we have people In our community who stand up 
and say, you people are playing the same old game. That 20% 
is the minority students, the poor white students, or the chil- 
dren who cannot read. I expect it is a non-acceptable kind of 
objective in virtually any school district if you have some 
people who have guts enough to stand up and say It. So we 
are going to say in our district-wide objectives (because I 
think they have to be saleable to the community) that all 
students will reach a specified level, knowing full well we are 
probably not going to reach that particular level. We are 
going to say on a district-wide basis that all students will 
develop communication skills. Then, on a school-by-school 
basis, we are going to be talking about certain numbers of 
students achieving at certain levels and we are going to try 
and communicate this on the basis that there will be continual 
improvement in those areas that have priority. That is a 
tougher sales job, but I think we owe It to the community to 
have them understand and accept the objectives. I think we 
have to put those objectives realistically in terms that people 
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understand and the people will buy. Now, I may not last as 
superintendent very long, but that is the direction we are going 
in. Those would be our school-wide objectives and what we 
have indicated for next year as an expectation is that each 
elementary school will have a school-wide objective. 

Now, again, we are in a developmental stage and in our 
district we have an elementary curriculum council and a 
secondary curriculum council composed of principals and 
teachers. We have asked each of these councils to be the 
prime agency in developing the district-wide objectives to 
take back and present to the Board. We have taken the posi- 
tion with the Board of Education that it is their role to set the 
district-wide goals and that it is our role as their managers to 
set the objective levels. What we are doing then is involving 
teachers and principals at these council levels to write the 
district-wide objectives which we will then bring back to the 
Board and ask if they are acceptable. If they say no, we will 
go back to the drawing boards, and try again. 

I am very hopeful that we do not get into a position where 
the Board of Education has to tell us what the priority areas 
are. I expect that there is a strong feeling right now on our 
Board of Education that we need to be doing something addi- 
tional in reading, but I have so far said let’s lay back and not 
blow the whole thing out of the saddle here because we want 
teachers and principals to be thinking about their own priori- 
ties at the present time. There is plenty of time then to say 
let’s emphasize reading this year or let’s emphasize human 
relations this year and so on. I do not think that there is 
enough time really in the day for every teacher in every school 
to zero in on every objective that exists in the district. I think 
that is very unrealistic with the kinds of tools we have to work 
with now. i also think, for example, that it is unrealistic to 
emphasize reading on a district-wide basis, because we have 
some schools up in the foothills where we have an average 
IQ of 110 where they are doing a great job in terms of com- 
parison to norm and so on in reading and we have some other 
schools where the average is years behind the norm. I do 
not think we need the same reading program. I do not believe 
we need the same emphasis on reading in these schools. So 
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what we are trying to do within “decentralization” is to say to 
each school that they must "do their own thing” and with their 
community. We are pushing very hard that each school have 
their own parent advisory group at the school, not the PTA. 

The program budgeting we feel is just a subsystem of this 
overall system and we will come to that within the next year 
in terms of having an appropriate program budgeting system 
that does lock into the objectives. We have a managerial con- 
cept in our district and we keep saying publicly that every 
person that is employed in our school district is competent or 
he would not be employed in the first place. It is the role of 
the administrators to continuously work to maintain and 
improve that competency. We say people are capable, we 
seek involvement of all our people, we like a free and open 
type of operation, we like to have a lot of flexibility. 

There are some other aspects of the management system 
that are very valuable, for it does clearly draw the line be- 
tween those people who are support services and those who 
have a responsibility to make things happen. We work hard at 
the fact that consultants have no role to play in evaluation; 
that consultants exist to meet the demands of teachers and 
principals. If the demand is not there, the consultants and 
support services will not exist in those particular areas. We 
have an instrument designed now and we are going out on a 
sampling basis and ask teachers and principals to evaluate 
the support services we have in the district. This will serve two 
purposes I believe. One, it will help us upgrade our support 
services and it will also help us to eliminate some of those 
support services which are not acceptable to the user in the 
field. 

The management system, we feel, does a good job of 
resource identification, both money and staff resources. We 
think it is a means of getting into performance contracting. 
We have some teachers this summer who have negotiated 
with us to do performance contracting internally by using 
the management system, by stating their objectives, and stat- 
ing their performance levels. We definitely see this as a 
vehicle to help us get into differentiated staffing in the district. 
We are not going to mandate any of these things, but they are 



coming from teachers who, as they get into the program and 
design it, recognize that maybe a differentiated staffing 
approach is a better process to reach particular objectives. 

When we have principals and staffs come in and talk to us 
about team teaching in ungraded schools, we say what is 
wrong with the way you are doing it right now. Do not come 
in and tell us you are going to start an ungraded school or 
study an ungraded program at an elementary school unless 
you come in and state some performance objectives and show 
us how the ungraded school or the team teaching process is 
going to help achieve better than what we are doing right now. 
There is no reason to rock the boat if it is not going to show 
some positive changes. You still hear people in the class- 
rooms continuously saying, “if I only had more money or if I 
only had a smaller class size.” Those are not acceptable 
justifications to me. More money to do what; smaller class 
size to do what? I think part of why we are suffering in the 
public eye is because we still continuously put the hand out 
and say if we only had more money we could do a better job. 
People do not believe that any more. 

The educational management system being developed 
by the San Bernardino City Unified School District is designed 
to aid teachers and administrators in identifying and respond- 
ing to student needs. It recognizes that student needs vary 
from time to time and place to place. There is no single 
instructional program which can meet the needs of all stu- 
dents at all times. Rather, it is important to remain constantly 
alert to changes in student needs and the accompanying need 
for revision and alteration in instructional programs. An effec- 
tive educational management system focuses on student 
needs and enhances the capacity of the organization to 
respond to these needs. In responding to the needs of stu- 
dents, the teachers can use the Educational Management 
System to communicate classroom needs and expectations to 
principals. Similarly, principals can use the Educational Man- 
agement System to convey building needs and expectations to 
the principal. We expect teachers and principals to mutually 
agree on objectives. Some of the most positive feedback I 



10 






/ 

/ 




/ 

/ 

/ _ 

/ 

i 

i 

1 

! have had this year is from teachers who say for the first time 

in their life the principal is sitting down with them and talking 
about instruction instead of proper room ventilation, chang- 
ing the tack-board display every two weeks, or having the 
chairs in a nice straight line. A lot of it still exists in our dis- 
trict; in our so-called observation summaries that are very 
much directed not toward what is happening to students but 
toward the principal’s impression of what the teacher is doing. 
The poor teacher is trying to role-play to the principal’s expec- 
tations when she does not know what the princip.al’s expecta- 
tions really are. 

We talk about mission, and the question we ask on that is, 
"If I were not here, what would not happen?” We try to get 
people to think about their missions as a principal, a dean, a 
counselor, or a teacher. 

In the areas of performance, “What are some areas where 
I perform essential tasks?” It is in this area I think we will get 
back to some analysis of function and roles people play. We 
very seriously want to have teachers define just what they do. 
We keep saying a teacher manages a classroom. Now what 
do we mean by managing a classroom? 

The third area, identifying key results, is our word for 
needs assessment basically. 

In the fourth area, determining performance indicators, 
we ask, “When the key result is accomplished, what sorts of 
things should I be able to observe?” 

Then, in the fifth area, we get into statement of objectives. 
What do you want them to be able to do at the end of the 
program, or what are the consumer’s requirements? 

Sixth, the development of the work plan and, within the 
work plan, the monitoring system. This would include the 
sequence of activities, what I will have to go through to reach 
the objective, and when the data will be collected. 

The last step is the performance review, which includes 
implications of the data for key results; what resources are 
needed? How can my supervisor help? What should be the 
priorities for next year? And what action should be taken? 
In our performance review which takes place on through the 
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year, we keep asking what kind of evidence you have that 
these things are happening. In the area of mission, we say this 
is the person’s essential contribution to the organization. 

In the areas of performance, we identify the categories of 
activities for which one is responsible. Where do I perform? 
Where should I be performing as a dean or as a director of a 
specific program or as a classroom teacher? In the key 
results areas, the intended accomplishments of highest 
priority are usually a demonstrable change in skills, knowl- 
edge or appreciations, and to identify an intended change in 
behavior. What do you want them to be able to do at the end 
of the program? 

Next in the area of performance indicators is the kind of 
evidence to be used in demonstrating the accomplishment 
identified in a key result area focusing on the key result; 
improved understanding of how our society operates and how 
to operate within the system. Some appropriate performance 
indicators might be the preparation of a notebook with 
research on at least two vocations or student planning of class 
schedules. Focusing on another key result: increased skill in 
paragraph comprehension. A performance indicator could 
be a Stanford Achievement Test, or paragraph comprehension 
teacher-made test. An example of our objectives would be: 

“Students will apply principles of set theory in 
solving additional problems by December 15, 1970, as 
measured by 70% of the children responding correctly 
to 80% of the items on the teacher-constructed pencil 
and paper test.” 

The development of the work plan and monitoring system 
contain the sequence of activities necessary to achieve the 
objective including, first, the identification of required 
resources and, second, milestones for collecting evidence 
to reflect interim progress toward objectives. I have had a lot 
of feedback from teachers saying that the most valuable part 
of what they have done is it has forced them to lay out a work 
plan with some dates and then to go back and check them- 
selves; which they see as a much greater value than the more 



common type of lesson plan that principals have for years 
required teachers to do. The performance review is a time for 
the principal to discover how he can help teachers reach their 
objectives. A conference for reaching agreement regarding 
needs and expectations, assuring of accountability, a com- 
parison of factual with intended results followed by identifi- 
cation of how the principal can help the teacher, and a time 
for setting new priorities is held. 

Let me conclude with some of the advantages I think we 
have seen this year in the use of this particular system. We 
have seen a great advantage in the use of this as an evaluation 
technique. I personally have met with each one of our sixty 
principals three times this year. I met with them early, nose- 
to-nose, shortly after the first of November and we discussed 
the objectives they had set up to implement with their 
teachers. Starting early in February and during the months of 
February and March, I went out to each school and spent an 
hour and a half to two hours with each principal reviewing 
their Educational Management System at the school. We went 
through and looked at their evidence and I asked the question, 
“how can I help you do a better job in the school?” Then here 
at the end of the year, mainly because I ran out of time, instead 
of going individually, I have had meetings of four or five prin- 
cipals and myself discussing and sharing their educational 
management system, discussing the pitfalls and what was 
good about their particular systems and what did others do 
in this particular area. So, it has been a good evaluation 
instrument for me and I have had good feedback from the 
principals. I mentioned a little earlier that I have had teachers 
who have said this has been the first time many of them have 
ever had the chance to sit down with their principal and talk 
about instruction and principals have done this many, many 
times this year with their teachers. They bring them in and 
talk about the objectives rather than the color of paper on 
the tackboard 

We have some real hangups and foresee problems in 
terms of determining whether or not an objective is, in fact, 
“a worthwhile objective.” If that is going to be a measure of 
accountability and a means of measuring for evaluation, how 
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do you decide that is really what ought to happen for that 
particular mix of students in that school? We are still wrestling 
with a good means of measurement. At the present time, of 
course, we have some mandated standardized achievement 
tests in California. We again are trying to show that we can 
make improvement even based on the use of standard 
achievement tests. But, we are hustling as fast as we can to 
get as many criterion-referenced tests developed in the 
district. 

We have used EPIC as an auditing firm for our ESEA 
Title I project. We are the orily school district in California that 
is having ESEA Title I educationally audit. We have gained a 
considerable amount from EPIC in terms of their relationship 
with our teachers and principals. We told the principals and 
teachers to ask the auditor all the stupid questions in order to 
help gain a better understanding of the management system 
and the writing of objectives. We think the audit is valuable 
to us as a learning situation. It is extremely valuable to those 
who are non-accepting, because you know the people in the 
community with whom I feel we have the least credibility are 
those whose youngsters are involved in the compensatory 
education programs. You can say that it is public relations 
activity, but at least we are going to be able to say here is 
somebody from the outside who came in and looked at our 
program. If they say it is bad, then we will have to buy that 
and improve it. If they say it is reasonably good, when you 
come in with your criticism, we will sit down and talk about 
where somebody else at least says we are doing a reasonably 
good job in these particular areas. 

I think I have already mentioned that the time involve- 
ment in this whole program is extremely frustrating to teach- 
ers and to me. Particularly when we deal with some people 
from the U. S. Office of Education who say that because of the 
fact so much money has been spent on planning so far 
without any significant implementation of programs. Federal 
money is not going to be available for planning — you are 
going to have to do your own planning, then come and get 
money for implementation. That is a hurdle, but we think we 
have some alternative solutions. It is going to be a major 
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hurdle in moving our management system ahead at any rea- 
sonable pace. The solution is free-up time of the experts, the 
teachers in the classroom, and the administrators, to become 
involved in program development. If we had the resources 
really to get some planning done right now I am sure we 
would move a lot faster. 

Another problem I think we foresee in the future is going 
to be the one I have already mentioned to you of relating the 
district-wide objectives for all students. 

In conclusion let me say that we do have a five-year plan 
in San Bernardino to the extent that we have done some com- 
prehensive planning within the management system ourselves 
and have implemented a systematic approach as to where we 
are and where we ought to be. 
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TEXARKANA DROPOUT PREVENTION PROGRAM 

MARTIN FILOGAMO, DIRECTOR 
TEXARKANA, ARKANSAS 

A lot of people have asked the question, “Why was Tex- 
arkana chosen to be the site of the first performance con- 
tract?” Well, it really was not anything magical. Texarkana’s 
Dropout Prevention Program is an ESEA Title VIII program. 
Texarkana does not have an outstanding dropout problem. I 
believe the dropout rate when we started the program was 
about 15-17%, and I do not know how this reflects nationally; 
but I know that other districts have a more severe problem. 
With the help of Model Cities, a management support group 
man by the name of Charles Blaschke originated the idea of 
performance contract using private industry in Texarkana. In 
September of 1969, the combined School Boards of Texarkana 
(Arkansas) and Liberty Eyiau (Texas) selected a contractor. 
They had many companies from which to choose: Radio Cor- 
poration of America, Educational Development Laboratories, 
Science Research Association, Westinghouse, Dorsett Educa- 
tional Systems, Learning Foundations, Edcon, and Interactive 
Learning Systems of Boston (Massachusetts). It was a very 
difficult task to select a contractor from this array of fine 
companies. 
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The most important thing that we wanted in Texarkana 
was a company which was technicaiiy sound. We wanted a 
program that wouid be easiiy accepted by the community, 
because Texarkana is rather conservative. We were aiso 
interested in getting a program that wouid be easiiy impie- 
mented into the Texarkana schoois because our iong-range 
goai was to improve the quaiity of instruction. We gave this 
between 35% and 40% weight. We were aiso thinking about 
price. We had $250,000 to operate a program under ESEA 
Titie Viii and we wanted to heip as many youngsters as 
possibie in grades seven through tweive. We decided to use 
reading and mathematics as the two areas of concern. We 
gave 35-40% weight to the price. We were iooking for a 
company that was wiiiing to come in and gambie. Since this 
was the first performance contract, many companies were, in 
fact, wiiiing to gambie and were wiiiing to iose quite a bit of 
money in order to estabiish this concept. We were aiso iooking 
for corporate commitment. We aiso iooked at things iike past 
performance and personnei. Bearing this in mind, the com- 
bined Schooi Boards selected the Dorsett Educational Sys- 
tems of Norman, Oklahoma, to operate our program. 

We dealt with youngsters in grades seven through twelve. 
Because of the very nature of the ESEA Title VIII legislation. 
Congress wanted to see results. They wanted to see actual 
figures on the reduction of dropouts. So we had to start where 
the dropouts really were. 

On October 15, 1969, the Dorsett Educational Systems 
opened the Rapid Learning Centers in Texarkana, Arkansas, 
and Texarkana, Texas. Originally, the enrollment was approxi- 
mately 150 youngsters who were two grade levels or more 
behind in mathematics and/or reading as reflected by national 
norm standardized tests. We used the Iowa Test of Basic 
Skills in Arkansas and SRA Achievement Tests in Liberty 
Eyiau, Texas. We also were looking at IQ — we limited the 
IQ’s to around 70. This was one of the inadequacies in our 
program since we used a standardized test IQ such as Lorge- 
Thorndike which told us if a youngster cannot read he cannot 
do this anyway. Fortunately, we have moved away from this 
and we are trying to use the Slawson Test, which is easy to 
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administer, inexpensive, and tells you about the same story. 

The typical youngster attends the Learning Center for two 
periods. He was usually taken out of his regular mathematics 
class and put into the Learning Center for mathematics. He 
remained on the bell schedule, which was approximately 
fifty-five minutes. If he was taken out of his English class or 
Reading class, whichever happened to be the case, and put 
into the Center for reading, he was in for the two periods. 
Sometimes we took them out of study hall. In some cases, with 
a doctor's excuse, the youngsters were taken out of Physical 
Education. The typical youngster attended the class, came 
into the Center, and punched in and out with a time clock. 
This served many purposes. First, we felt it would be good to 
introduce them to a time clock; they would see how maybe 
their father or mother punched in and they would be doing 
the same thing. Second, it gave us a good attendance check. 
Third, we were able to ascertain the number of minutes the 
youngster participated in each program. 

In Phase I, which was the first year of the program, the 
Dorsett Educational Systems used primarily the teaching 
machine M86, which was manufactured by them with pro- 
grammed, self-instructional material consisting of film strip 
and a record that were synchronized. The youngsters partici- 
pated in these programs as they were listed. They could 
ascertain the progress since most of the programs were in 
some sort of sequential order. So, if a youngster took lesson 
4B, he naturally assumed that lesson 4C would follow. As the 
youngsters came into the Center they went to their individual 
folders and received the instructions for the day. The Center 
was staffed by one certified teacher and one para-professional 
for each sixteen students. We tried to arrange the Centers so 
no one would be looking over someone else’s shoulder. 

Once a youngster completed a program (which took 
about twenty minutes), the instructional material of the Dorset, t 
program was synchronized with buttons to be advanced if 
they pressed the correct button. If an incorrect response was 
given, the program stopped. This was one of the inadequacies 
in the program in that the program stopped and did not say 
anything so the youngster naturally assumed that he was 
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wrong. If he was industrious enough, he would try to flip the 
machine back and find out what happened. If he was not, he 
would just press another button and if it did not move, he 
would press another until it would move. In order to compen- 
sate for this inadequacy, the progress checks were given 
immediately upon completion of a program. If a progress 
check was taken and if a grade of 80% or better was not 
achieved, then the youngster was not able to continue on in 
the cycle. If he did make 80% or above, then he entered into 
one of the most controversial parts of our program called 
contingency management. 

When I started in the program, I knew what contingency 
was and I knew what management was; but I did not know 
what they meant together. I was soon to find out. If a youngster 
made 80% or above, his first reward was S & H green stamps. 
As I have made these presentations throughout the country, 
when I mentioned S & H green stamps, I always got four or 
five snickers from the audience. However, the stamps did 
turn the youngsters on because it was a novelty. Some of the 
girls wanted to get hair driers, so they set their goal. Most 
of the boys, after two or three or four months of S & H green 
stamps, threw them in the wastebasket or gave them to the 
girls. But you know, if industry can give S & H green stamps 
as an incentive, I wonder why the schools cannot. I know that 
if a first grade teacher gives a gold star to a youngster for 
completing a successful! assignment, I am not sure there is 
very much difference between S & H green stamps. I do not 
know whether it should be S & H green stamps, a hamburger, 
or a pat on the back. It is our responsibility to inspire these 
youngsters and, much to my chagrin, the instructional material 
that most of us use certainly would not be conducive to 
inspiring. I do not know whether we can justify contingency 
management in that area. Contingency management, as I say, 
could be many, many things; but whatever it takes, I hope we 
will do it. 

As tests were given (and this is history now), many of the 
people on the ESEA Title VIII staff evaluated them and dis- 
covered that there were too many actual test items in the 
instructional program to be very beneficial to the youngsters 
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as far as learning is concerned. In other words, we felt that 
Dorsett was teaching the test rather than "teaching to the 
test.” 

The Iowa Test of Basic Skills was used which tests knowl- 
edge of Roman numerals. I am not too sure that a potential 
dropout needs to know anything about Roman numerals. But 
they had to teach the Roman numerals because they knew the 
students would be confronted with Roman numerals on the 
test. Because of this, the evaluator and the auditor decided 
that they were not able to measure the test results since the 
contamination could not be traced as to when it started. Then, 
all the results were left out in the cold. This does not mean 
that there was not any achievement; because upon visiting the 
Center many times, you could tell that the youngsters were 
learning. Because many of the youngsters, instead of accept- 
ing rewards such as free time to listen to records or to play 
checkers or to read magazines were telling the teachers, 
"Please don’t bother me right now. I’m going through this 
program and when I get through with this program I’d like 
you to give me another one.” When you get this type of 
reaction, you know that something is going on. 

Well, at the end of the first year we were confronted with 
either dropping out of the program or continuing with another 
contractor. We decided to go on with another contractor, so 
we went through the Request for Proposal bit again and the 
School Board selected EDL (Educational Development Labor- 
atories), a division of McGraw-Hill. Their program is intensive 
and uses one teacher and one para-professional. EDL 
increased the class load to twenty students, and is operating 
on a point system. They guarantee for about 130 days of 
instruction a 1.0 to 1.9 grade level increase. If a youngster fits 
into this guarantee, 75% of the payment would be on a norm- 
reference test and they would receive about $33. Twenty-five 
percent of the payment would be from a criterion-referenced 
test. A criterion-referenced test, as you have read in the last 
few months, is just a glorified name for a teacher-made test 
except that it may be a little more sophisticated. Our criterion- 
referenced tests were based upon EDL’s objectives. For each 
cycle of learning, EDL has objectives. We have asked the 



company to submit a pool of five times as many items as we 
will need in math and reading. From this pool of five times as 
many as we need, the evaluators randomly select one out of 
five. 

We did give a test on May 4 and 5, 1 971 , and, although the 
final results are not in and are not verified, they look very 
good. Last week we gave our test for the 75% payment which 
was an Iowa Test of Basic Skills Form 5 for the pre-test (and 
Form 6 for the post-test). These forms are in the norming 
process and are not available to schools and will not be 
available until September, 1971. 

One of the unique aspects of Phase II of our program is 
that we have endeavored to turnkey the successful compo- 
nents of Phase I into the regular classrooms of grades seven 
and eight in mathematics and reading. What we said is that 
we believe that teaching individuals with individualized 
instruction is the best way to learn and the best way to teach. 
So, we took the components of our Phase I which we thought 
were successful and put them into the turnkey classes. In 
September, 1970, we started our in-service training with our 
turnkey teachers. Many of these teachers were very reluctant 
about going into this. Some of the English teachers said they 
had never taught reading before and they were afraid. But 
after much pleading, we got together approximately sixteen to 
eighteen teachers. They received excellent training for two 
weeks. The material that was ordered did not come in. You can 
imagine how the teachers felt when they did not get the 
material. They said this was just another one of those pro- 
grams we have heard about that nothing ever comes of it. 
Finally, the material came in November and the turnkey 
classes started. The turnkey students were also pre-tested 
and post-tested, so we are going to take a look at what they 
have been able to do. 

One of the most rewarding things that has happened to 
me this year is that four or five of the turnkey teachers (in 
fact, the ones that were most vociferous in their demand not 
to be turnkey teachers), told me that they thank me for the 
opportunity of becoming turnkey teachers. They had heard a 
lot about in-service education, they had heard a lot about 
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individualized instruction, and they had heard a lot about 
motivating students. They were able to really implement this 
and they were able to see these youngsters who were in the 
turnkey classes. The youngsters in the turnkey classes were 
the youngsters in the Learning Center last year who were 
termed successful. I do not know how you can term anyone 
successful if you say that the results were invalid, but anyway 
we did. The youngsters who were not successful were placed 
in the Learning Center again. 

What it did show us is that the youngsters who did partici- 
pate in the turnkey classrooms also participated more in other 
classrooms, such as social studies, health and science, than 
ever before. I think as we talked with them we were able to 
ascertain that the turnkey classes gave them more confidence 
than they had ever had before because there was not any 
pressure to learn. We did reduce the dropouts in Phase I from 
14% to 8%. 

If you direct yourself to the target population of young- 
sters who were two grade levels below and had an IQ of 75, 
the dropout rate was about 1.4. We are talking about young- 
sters who were on the verge of dropping out. They just had 
not been that successful in school and, in some cases, the 
teachers wanted to get rid of them. We really do not know 
what will happen this year as far as achievement. We know 
that our dropout rate at the end of the semester was less than 
2% . There are a lot of youngsters who drop out in the spring- 
time; but hopefully, we will be able to keep the dropout rate 
down around 5% for our target population students. 

We are now entering into Phase III. We are operating a 
five-year program. Next year we are going into performance 
contracting once again with EDL. We are going to expand our 
program with the help of Model Cities and we are going from 
K-12 with Learning Centers in the elementary schools, and 
(as they exist now) in the secondary schools. We are really 
not trying to fool ourselves into thinking people drop out just 
because they cannot read and write and figure. They drop out 
for many varied reasons. 

Next year, we are going to go into the home a little bit 
and talk to the parents and get them more involved in the 
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program. The parents have been involved in the program this 
year, but not to the extent that we would like to have them. We 
are also going to try to individualize instruction more and 
more. This is going to be done through staff training and 
through the acquisition of materials. We are also thinking 
about revising the entire curriculum (K-12), with the help of 
the staff training component. We are also going into the area 
of vocational education simply because dropout prevention 
and vocational education go hand-in-hand. 

This may be a little too much to bite off; but at least we 
feel we have our foot in the door because we have the super- 
intendents behind us. The principals are behind us. In two 
surveys we have taken (one last year and one this year), 86% 
of the teachers are behind us. We feel that the days have been 
rocky. This year was much better. Hopefully, next year will be 
even more significant and it is our hope someday to develop a 
dropout prevention design that could be used in Seattle, Bal- 
timore, Tucson, or anyplace else. This is one of the hopes of 
the U.S. Office of Education and the Dropout Program — to 
have a program that will work and that other school districts 
can pick up. 



GRAND RAPIDS PERFORMANCE 
CONTRACTING PROGRAM 

JOAN WEBSTER, DIRECTOR 
GRAND RAPIDS, MICHIGAN 

In Grand Rapids, we view performance contracting as 
being just one step in bringing about accountability by chang- 
ing strategy, and assessing current programs and needs, then 
deciding where we go next. Technical assistance is talking 
about management support systems, getting help in writing 
Requests for Proposal, using some other people to help you 
do your thing. In Grand Rapids, we did not bother with a 
Request for Proposal. We telephoned three companies that 
were willing to sell us a program and we just went out and 
bought them. We tend to always put the cart before the horse. 



but we have come out all right. We think the Request for 
Proposal is a dead issue. With forty-seven companies on the 
market, we think we have developed enough internal sophisti- 
cation to find a program that met what our needs were and 
anybody who wants our money can rewrite what they already 
have. But we do not have the money to hire a management 
support group. From what we have heard about management 
support gi'oups, we are not too sure we really want any of 
them writing our Request for Proposal. That is why I would 
like to see my district go through the kind of process you have 
been going through here at EPIC so they would know what 
performance objectives are. Then, they could probably write 
their own. 

We are also putting in MBO, which is management by 
objectives in our system. We are starting to direct our admin- 
istrators to work with PERT charts and critical path move- 
ments. 

Performance objectives is just the specification of per- 
formance objectives in measurable terms. Program budgeting, 
again, is just PPBS or the allocation of resources in accord- 
ance with your program. Performance contracting to us is 
just measuring outputs instead of inputs. Staff development is 
the pre-service that the companies give to our staff in-service 
and the constant monitoring they give to make sure we do not 
change their program because we (as people in the school 
system) tend to take the easiest way out. Evaluation is just 
continuous monitoring, data collection, data analysis, report- 
ing, and dissemination of information. Cost effect of this is 
really dearest to my heart because I think this is going to be 
the next important thing to come out of performance contract- 
ing when we start taking grade gains that students make and 
applying a cost factor to them and deciding what are the 
sensitive trade-off factors. If I put x number of dollars here, 
will that assure me that I am going to get more grade gains 
here? We have a research study going on in Grand Rapids 
now that is trying to identify some of these things. Program 
auditing we look at as being performance control based on 
external reviews. 

We have three contracts: Alpha Learning Systems, West- 
inghouse Learning Corporation, and Combined Motivation and 
24 



Educational Systems. The Alpha System is the Office of 
Economic Opportunity program. It is one of the twenty 
research programs. In this program, we are hampered by a 
lot of constraints put in by the Federal government; the biggest 
one being that we have to put first graders in this program. 
All of the students in our programs are two, three, and four 
years behind in reading and mathematics. If a first grader 
comes to your class in September and you look at him and say, 
“Sweetie, you are two years behind,” that could really blow 
someone’s self-concept. So, we feel one of the biggest weak- 
nesses in the Office of Economic Opportunity program is 
including first and second grade students. We think there 
should be a preventive program — or just the best teachers 
in the world in the first grade — and forget a remedial per- 
formance contracting program in the first or second grade. 
The Alpha program has six hundred students from grades one 
through three and seven through nine, with seven certified 
teachers and eight para-professionals. The students are taken 
out of the classroom into a learning center. The Alpha pro- 
gram is all software. Program instruction materials are from 
some twenty-five different publishers. 

The teachers are taught to diagnose the students and 
prescribe the curriculum that the students are supposed to 
follow. Our question on this program is, “Are teachers really 
ready to diagnose the student difficulties after just a two-week 
pre-service?” 

The Westinghouse Learning Corporation, which is now 
the Learning Unlimited Contract, has 220 students from grades 
one through six, in two buildings, with two certified teachers 
and eight para-professionals. This program is what we call 
midware. There are tape recorders, carpeting, air condition- 
ers; but again, twenty-five different resources to use. 

The Combined Motivation and Educational Systems 
(CMES) program contains 590 students from grades six 
through nine, four teachers, and eight para-professionals. It 
is all hardware. They use the System 80 machines, the Huff- 
man machines, the language master, and some others. 

The Alpha program was funded by the Cffice of Economic 
Cpportunity. Westinghouse is funded by a combination of 
ESEA Title I and local monies and Combined Motivation and 
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Educational Systems is funded by Model Cities and local 
monies. Our Model Cities people are quite happy with this 
program because it is the "tinsel” program and there is a lot 
of hardware. Well, we are kind of pleased with the program 
for a different reason; that being we put in fifty special educa- 
tion students who were not identifiable to the contractor. The 
contractor has not been able to successfully identify all fifty, 
which makes us wonder how we place students in special 
education in the first place. In the second place, what are we 
going to do about it? So, next year we are going to have a 
performance contract for educationally mentally retarded 



rogram with it. 

another year 
ager and they 
the important 



students. I think we will be using the CMES pr 

We are sticking with the three companiei 
simply because all three have an on-site man 
are cutting down on our costs. We feel one o: 
variables in the programs in our school is to hav'p these on-site 
managers who are not tied to the school system, but who are 
responsible for living up to their side of the contract and who 
make sure the teachers are using the system as it has been 
designed. 

On the Alpha system, the students go through a flow 
chart. After they are tested and diagnosed by the teachers, 
they are put through a flow chart which tells them what to do 
in the workbook and how they are rewarded. If they can 
achieve at an 80% level, the teacher will give them as a token 
economy a metal washer which we call a "skin,” an acronym 
for "supplemental knowledge incentive note.” The younger 
children get these washers to wear around their neck on a 
leather string or to carry in a baby food jar. The only person 
who is upset with the washers is the man at the local gas 
station because they fit into his gumball machine and they 
are worth two-for-a-penny. The younger students get to take 
these washers to the reinforcing events room and trade them 
in (with the para-professional) for some game they want to 
play. It has brought about some interesting kinds of things in 
that children are learning to add and subtract right there in 
the room. They will say, "Hey, it takes ten skins for this game. 
I have two, you have eight; let’s put them together and we can 
play.” 
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This idea of awards was one of the biggest criticisms we 
had. Were chiidren aiways going to expect a reward? We find 
that some chiidren are now saying: “Boy, i just feei so good 
about myseif and what i am doing.’’ Even though they are a 
ninth grade chiid working on a fourth grade ievei, they are 
piaced properiy in the materiaisand they feei good about it. 
They say, "Okay, teacher, i am going to contract with you. if 
i work four days in a row for both of my periods (because the 
junior high students go for two back-to-back periods of fifty- 
five minutes each), “can i have the fifth day off?” Now this 
took a iittie doing to move the school system off dead center 
on this, but we convinced them by expiaining that it is in the 
contract: that you are going to iive up to the proposai Aipha 
wrote. 

That is the heart of one of the probiems we are having 
right now. Nobody ever read the proposai either and we 
accepted the proposai as Aipha presented it— -i am teiiing 
you peopie what to avoid. We accepted the proposai as Aipha 
wrote it and it is in there that, as weli as student incentives, 
there wiii be teacher incentives based on the interim perform- 
ance objective test. And there was a budget in there (i. e., 
$3,000) for teachers and para-professionais. But nobody saw 
this on page 92. i thought they aii saw it. So, now we have a 
grievance fiied against us by a iocai union because we ietthis 
outside agency pay bonuses to the teachers and we openiy 
and fiagrantly vioiated the K-12 master agreement. 

The union is asking for three things. They want the direc- 
tor fired. They want reiief paid to the other 1,902 teachers, 
which wouid cost about $750,000. They aiso want Aipha 
thrown out. Weii, obviously, we are not going to lose and 
neither is the union. Keep watching the newspapers because 
we are going to take it on in the nationai interest. We feei very 
strongiy about it. I cannot teii you what we are going to do, 
but keep reading because we are going to do it. 

When the junior high students get into the free room, 
they can do some unusual kindsof things. They piay pooi, pin- 
baii, dance to the jukebox, drink Coke. The oniy thing we 
couid not get permission for was to smoke cigarettes. We do 
have a pooi tabie in there, if we in Grand Rapids had moved 
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in a pool table, the superintendent would have gotten hung. 
When the community started screaming about it, we said, 
“Well, we have a contract. Your advisory committee approved 
this. So now we have our pool table.” But a funny thing hap- 
pened. The CMES program uses achievement motivation 
sessions as a reward and they did not have a pool table and 
you are dealing with the same age group. Some of the children 
that live in the area where the CMES program is are fostering 
our illustrious master plan over to another area where the 
Alpha program is and they get to shoot pool. They go back to 
their home neighborhoods thinking, “Man, they get to sit and 
rap with the teacher, they get to shoot pool,” so now CMES 
has a pool table and a pinball machine and some other kinds 
of things. 

I thought you might be interested in what the prices are 
in the Alpha program. If a student does not achieve one year’s 
grade gain, we do not pay the contractor a thing. If he 
achieves from one to 1.24 year’s gain, we will pay $75; this is 
per subject. Now we are talking about help with the pre- and 
post-testing. We are because the Battel Memorial Institute 
came in to do the Office of Economic Opportunity testing and 
we are just hopeful we can duplicate the chaos we had in the 
fall so we have some reliable results. We are doing a cross- 
evaluation and we did another set of testing with the students 
in evaluation design and we hope to be able to have some 
credible measure to be able to determine whether or not the 
Battel testing was honorable, realistic, valid, reliable. We are 
concerned with testing, but there really is not any better way. 
We tested over sjx thousand students in the district — all of 
the students in the three programs plus students in the 
Project Read and in control groups and in traditional remedial 
reading. This is what we are applying our cost factor to. We 
want to know what we are spending and what we are getting 
for our money. The only way we are going to do it is this way. 

There may be a lot of people up tight when we are fin- 
ished. One of the groups, I think, is going to be up tight if I 
talk about traditional remedial reading at this point in time. We 
put a half-million dollars into this program; it does not get 
under way until about the fifteenth of October when the stu- 
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dents are identified. Teachers will work with four students for 
an hour and we really have no measurable gain. Teacher cost 
here is very high because our very best teachers are in tradi- 
tional remedial reading. I do not know if it is the district’s fault 
because they do not give the resources to work with. Maybe 
they do not give them the help in designing good programs, 
or maybe it is just the way we handle the students. It is a pro- 
gram the students say is for “dumb-dumbs.” The students do 
not like the stigma. Now, in the contract learning program, the 
students think they are pretty cool to be in it. We had two 
schools near the Westinghouse schools and, prior to this 
time, the schools could determine how they wanted to spend 
their allocations and they always added more para-profes- 
sionals or another teacher and still no results. This year, these 
two schools went to their principal and said, "You know, we 
like that program in the other two schools.” We did put it on 
a non-contract basis. In a way, we are already turnkeying 
by buying the system. My own opinion on turnkeying at this 
point in Grand Rapids is that we are not sophisticated enough 
to do it. We think we have learned a lot, but unless we have 
this outside catalytic agent, we are going to be back on dead 
center sooner than we got off from it. We tend to use the 
easiest way out. We tend to write off students and write off 
programs, so my recommendation is that we go another year 
with all three programs. 

The Westinghouse program gives their own diagnostic 
tests to students and then Telex it to Albuquerque. You could 
buy the system for the Telex machines. They put these 
modules together. Four million dollars in research went into 
putting modules together like this and there are just thousands 
of modules. But if a child should miss two items within a sec- 
tion on the diagnostic test, it means that he needs this kind 
of thing to work with. When the student gets his module, it 
says to take the pre-test. This is the same kind of a test a 
teacher would construct, so it is nothing earth-shaking. It has 
questions such as: How many dimes equal one dollar? How 
many nickels equal one quarter? If a child should success- 
fully pass 80-90% of these items, he does not have to work 
with the module. But if he does not, he works through it and he 



goes to number one, and it says “do learning system prepara- 
tion mathematics money number 1030” (all these books are 
coded around the room). The child gets the activity sheet and 
cassette and puts on his earphones and goes to it. When he is 
finished with this, a para-professional or the teacher will 
check it. If he achieved a 90% or better he gets the “A.” If he 
did not, he gets an “I” and has to review it. It is initialed by 
whom it is checked, so the students here can also contract 
for an activity and go to a free room (they call theirs an activ- 
ity room) and they get “buzz” breaks. One “buzz” break is 
five minutes and two “buzz” breaks are ten. Their reward is 
based simply on achieving an “A” in the lesson. 

In the Alpha program, it is based on achievement in terms 
of learning something and getting a 90% or better score. Our 
use of contingency management in the Westinghouse program 
is the form of having a card on top of the student’s folder. If 
the teacher sees them doing something that she approves of 
she can mark her initials on it. When the child gets forty marks, 
he can exchange the card for a coupon. The coupon can be 
exchanged for a taco, a hamburger, a movie, or a bowling 
game. We have been severely criticized for bribing children, 
but we do not see anything wrong in that because we feel this 
way. If you say, “Look now. You study all year long and I will 
prss you to the third grade,” the child will say, “Oh, whoopee. 
What’s so great?” Or if you say, “You work hard for four years 
in high school and you get to go to college,” this student we 
are dealing with looks at us like “College! Like what is it? That 
is out of sight for me.” But if we can give rewards immediately 
to children, they are motivated. It works for us. I am not sub- 
scribing to it for everyone; it might not work for you. 

The Combined Motivation and Educational Systems pro- 
gram deals a lot with achievement motivation. This is because 
Walt Thomas, who wrote the program, is the ESEA Title III 
values man. They go through six steps in working with the 
students in what they call rap sessions. Once a week the 
students get to sit down with one of the highly-trained teach- 
ers and just talk about things that bother them. These young- 
sters are all thirteen to eighteen. They have some real prob- 
lems. Nobody ever talks to them. Nobody really relates to 




30 



them and nobody cares. But, you get six girls together in a 
room with their teacher and they start talking like. “You 
know, I have a problem with my boyfriend. He is walking some 
other girl home. How do I handle this?’’ or, “1 cannot get a 
babysitting job. Nobody wants me.’’ or, “They cannot afford 
me.” or, “I do not know how to go about it. What can I do? 
‘‘Gee, I do not know what I want to do this summer.” So, the 
teacher will talk to the youngsters about these kinds of things. 
Somebody is finally paying attention to them. They like it. The 
thing that we feel the most about performance contracting is 
that it brought about internal reform to our system. It not only 
made the teachers and the students accountable, it made all of 
us in middle management and the central office accountable. 
We have done some pretty rotten things in the past like take a 
first grade teacher, put her in a room and tell her, “Honey, 
here are your Dick and Jane books. Now go to it and I will 
check with you in June when you fill out all your cumulative 
records,” and that is it. Now what we are saying is, “We are 
going to stop by every week and monitor what you are doing 
and we are going to help you. If you need anything, come to 
me.” 

One thing I saw happen in the Alpha program was that a 
teacher wanted a book. She was not getting to the youngsters 
in mathematics and she talked to the contractor about it and 
two days later she had a different book. When I taught in the 
first grade and I wanted another book, I would fill out a pur- 
chase requisition: it went to the vice-principal, then to the 
principal, then to the purchasing man, and I do not know how 
many other people, but I always got a note back saying, “We 
will order it in March for next year.” It did not help the child 
this year. Now the contractor knows he can make $50 on this 
student if he really gets through the course. So, he goes out 
and spends $4.95 and gives the teacher the book and every- 
body is happy. It has kind of shaken up our school system. I 
think it is kind of gutsy of our school to go into three contracts, 
but here was a chance to do something on somebody else’s 
money. Our taxpayers are voting down millage issue after 
millage issue, and, by the way, this year is the first time in 
twelve years we passed a seventeen mil issue on the first 
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ballot. We still cannot believe it either and we only passd it 
by about 2300 votes, but we passed it. We are kind of excited 
about it and we think it is because we are doing something 
in the schools. 
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ELEMENTARY LEVEL, VALLEJO, CALIFORNIA 

DOUGLAS H. STELLN^R; PRINCIPAL 
JOHN CARTER, TEACHcR 
VALLEJO CITY UNIFIED SCHOOL DISTRICT 
VALLEJO, CAdFORNIA 

--^'^fgure 1 illustrates a simple model of a system outlined 
with an environment. As we examine the background of our 
district’s performance-based project, I would like to have you 
consider it as the Vallejo Public Schools. This system is a 
combination of interrelated — and interacting — parts, 
designed to achieve a common purpose. That is, to provide 
its students with the technical and regulatory skills necessary 
for them to become productive members of society. As is the 
case with all "systems," the Vallejo Public Schools system is 
concerned with environments — both immediate and greater. 
These environments can be considered as a complex of many 
components which act upon the system and have an effect 
upon its structural, functional, or for that matter, Its survival 
characteristics, 

Historically, as we view the system’s internal changes 
leading to performance-based experimentation, we believe 
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that it is possible to isolate five distinct, yet inter-related, 
environmental changes which brought about this transmuta- 
tion. We have chosen to identify four of these factors as infor- 
mational input, which provided the system with the essential 
ingredients for decision-making. In turn, we have labeled 
them as: 

DECENTRALIZATION 

ACCOUNTABILITY 

P.P.B.E.S. 

ACHIEVEMENT DISSATISFACTION 

Decentralization 

First, there is what Stanford's Luvern Cunningham refers 
to as “The Magnificent Pandora of Decentralization.” This 
“informational input” was initially injected into our system in 
the form of a 1968 report by Dr. Bernard McKenna. Dr. 
McKenna had been commissioned by the District to make an 
assessment of Senior Administrative Staff arrangements, and 
had recommended a revision of those positions and their 
relationship to the district as a whole. The recommendations 
were adopted and the new form, or organization, was imple- 
mented in April of the same year. In the final analysis, we 
believe that this adoption and implementation was intended 
to decentralize decision-making, and to provide encourage- 
ment for field administrators to seek out ways to experiment 
and innovate. 

Accountability 

Viewed in perspective, it seems obvious that the new 
education “in" word — accountability — was closely linked 
with our district’s concept of decentralization. Dr. John Nicoll, 
our City Superintendent of Schools, made this emphatically 
clear in a July, 1968, memorandum addressed to District 
Administrators. In clarifying the above-mentioned report he 
stated, and I quote: 

"Each principal is earnestly requested to accept the 
responsibilities involved in the new organizational pat- 
tern. Remember that it is intended to decentralize 
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decision-making as much as possibie and encourages 
administrators to make as many decisions as feasibie 
within their own areas of responsibiiity and with due 
regard fcr board poiicy. Centrai office administrators are 
avaiiabie for advice but principais are expected to 
assume the responsibiiity and the ACCOUNTABiLiTY for 
decisions concerning their own schoois." 

Other voices aiso were providing credence to this con- 
cept. For exampie, there was Leon Lessinger who promoted 
educationai accountabiiity during his 1968-70 tenure with the 
U.S. Office of Education. Certainiy, Wiison Riies used the term 
repeatediy in his campaign oratory and has continued to 
espouse this concept since his eiection. 

Finaily, President Nixon’s March 3, 1970, Speciai Mes- 
sage on educationai reform provided executive prestige to 
this concept with this statement: 

“Schooi Administrators and schooi teachers aiike 
are responsibie for their performance, and it is in their 
interest as weii as the interests of their pupiis that they 
they be held accountabie.” 

P.P.B.E.S. 

Weicomed by some, damned by most, and ignored by 
many, P.P.B.E.S. became informational input some four years 
ago. 

In 1967, the California Legislature enacted Assembly Bill 
61 which authorized the Advisory Commission on School 
District Budgeting and Accounting. Under the direction of this 
Commission, a plan was prepared to Implement planned, 
programmed budgeting throughout the State. 

As early as 1967, selected Vallejo personnel were In- 
volved In systems training and, by the winter of 1970, approxi- 
mately eighty teachers and administrators had received 
instruction In, or exposure to, this concept. 

Presently, the most accurate timetable available calls for 
the Implementation of P.P.B.E.S. In all California School Dis- 
tricts for the 1973-74 fiscal year. 
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Achievement Dlseatlstaction 

We have elected to classify the final element of informa- 
tional input as Governing Board dissatisfaction with student 
reading achievement in the primary grades. 

Directed to the eiementary schoois within the system, 
and perpetuated, no doubt, by evidence of nationai concern, 
this dissatisfaction peaked in the Winter of 1970 when the 
Governing Board was presented with a report of the 1968-69 
State-mandated test resuits. Foiiowing the report, the Board 
members expressed their concern with Vaiiejo’s iow ranking 
and directed the staff to make a determination of what couid 
be done to raise the scores on future tests. In March of the 
same year. Board members and the Superintendent piaced 
further emphasis upon their expressed concern by ranking 
reading skills development as the school district’s Number 
One goai for the 1970-71 school year. 

In our opinion, the fifth and final environmental change 
leading to performance-based instruction can be identified 
as the valued societal target of the right to read. 

Targets are defined by systems analysts as societal 
objects toward which system goals, objectives, and output 
should be directed; and this particular target needed minimal 
translation. As a matter of fact, James Allen’s famous Sep- 
tember 1969 address could not have been more explicit than 
when he stated: 

"We should Immediately set for ourselves the goal 
of assuring that by the end of the 1970’s the right to read 
shall be a reality for all. That no one shall be leaving our 
schools without the skill and the desire necessary to read 
to the full limits of his capacity.’’ 

In an age which is characterized by a perfection of means and 
a confusion of goals, it was refreshing to find a target that was 
so easily Identifiable. 

At this point, it is obvious from an examination of our 
model that those five environmental changes were well 
positioned around our Vallejo School System in the early days 
of 1970. Further, it should also be obvious that if the system 
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were to remain in harmony with its environment, it would be 
incumbent upon it to consider a change in the direction 
advocated, and likewise to produce an OUTPUT that would 
be directed to the target. Plans to initiate such internal 
changes took place in the Spring of the same year. 

As has been previously noted, primary reading improve- 
ment was elevated to the top position in the district’s priority 
rankings of program goals. 

Additionally, Dr. Guillermo Lopez, Associate Superin- 
tendent for Instruction, and all elementary principals were 
directed to develop a plan which would include (1) specific 
objectives, (2) procedures, (3) a primary reading program 
budget for 1970-71, and (4) an evaluation component. Two 
and one-half months of broad-based participation and pro- 
tracted discussion were to follow before such a plan could be 
formulated and presented to the Governing Board in June. 

Finally, however, on the evening of June 2, 1970, 
Performance-Based Instruction arrived in Vallejo, California. 
Additionally, on that same evening, sixteen Vallejo elementary 
principals fell heir to an accountability expectation and all of 
its related administrative paranoia. However, when viewed in 
retrospect, it is obvious that the final plan was certainly in 
keeping with that informational input depicted in our model. 

Additionally, it has been pleasant to discover that the 
district plan also encompasses what Leon Lessinger is now 
advocating as the Three Basic Rights of Modern Education. 
For the benefit of those who may not be familiar with Les- 
singer’s new book, may I quickly paraphrase: 



RIGHT #1 

The right of each child to be taught what he needs to 
know in order to take a productive, rewarding part in society. 

RIGHT #2 

The right of the taxpayer and his elected representative 
to know what educational results are produced by a given 
expenditure. 
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RIGHT #3 

The right of the schools to draw upon talent, enterprise 
and technology from all sectors of society instead of being 
restricted to their own overburdened resources. 



PART II — Davidson Program Structure 

The L. E. A., in this case, the primary staff of the John 
Davidson Elementary School, when confronted with a man- 
dated verifiable performance objective, had numerous alter- 
natives: Early retirement, change of assignment, dissent, 
nurturing the hope that accountability is another passing fad, 
or to stand up and be counted. The Davidson staff selected 
the latter course. 

After many hours of deliberation, a decision was made 
and concensus attained. The members of the staff were 
adamant in the belief that they knew how to teach reading, 
but were equally certain that they lacked the expertise 
required to write a proposal which would specify a verifiable 
performance objective and the concomitant process objec- 
tives necessary to meet the demands of the district. It was 
with in-service education funds available that a contract was 
negotiated with EPIC. 

The staff met with the EPIC Consultant for two days 
during the Summer vacation. A proposal was written and 
ground rules specified for an in-depth workshop to be held 
before the Fall school term commenced. During the forth- 
coming session, the staff would become involved with the 
process of writing verifiable student performance objectives 
and also develop a monitoring system (see Figure 2 for Imple- 
mentation schema). The EPIC Consultant, as well as providing 
in-service education, also became the external evaluator of 
the John Davidson Primary Reading Program. 

The staff, during its initial deliberations, also considered 
the budget and identified the need for a team leader — some- 
one who could coordinate the program. The team leader’s 
responsibilities were delineated in the proposal and consist 
in part of providing ongoing in-service education pertaining to 
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Figure 2. 
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the program, a weekly monitoring of the program and coi- 
iecting data. 

During the schooi year, the Consuitant conducted two 
on-site interim monitoring visits of the monitoring system and 
one on-site monitoring visit of the data. The finai report wiil 
be presented not oniy to the staff but the Governing Board of 
the VCUSD. At this time, the John Davidson staff wiii be in the 
enviabie position of determining the effectiveness of the 
program and recyciing for the next schooi year. 

The preceding has been an attempt to briefiy portray the 
‘‘WHY’’ and “HOW” of the John Davidson encounter with 
accountabiiity. Before embarking upon a review of the moni- 
toring system, it behooves us to ask the question “What is a 
monitoring system and why is it needed?” 

The concept of accountabiiity demands that three condi- 
tions be met: The specification of the desired outcome, the 
initiation of a system of auditing and monitoring, and the fiiing 
of a pubiic report. The monitoring system ensures that accur- 
ate conciusions may be drawn with respect to what process 
has resuited in the desired change in behavior or why the 
outcome is different from that expected. Further, a monitoring 
system provides information about the interim objectives of 
“stepping stones” which iead to the fulfiiiment of the finai 
objective and suggests changes for improvement. 

You may recail that in the EPiC Scheme for Evaiuation, a 
monitoring system was specified as a component of PHASE i 
— The Planning Phase. The schema was designed primariiy 
for use in the deveiopment of an education program. How- 
ever, with siight modifications of the terminoiogy in PHASE i, 
this schema was easily adapted to use at the classroom level 
(Figures). Basically, the modified PHASE I identifies the data 
which will be monitored. Phase I, the planning phase, con- 
sists of three steps. 

At Step One, the teacher identifies and describes the 
variables. Basically, this consists of deciding the specific 
content area to be considered since the teacher presumably 
has made some sort of needs assessment or diagnosis of the 
members of the class. 
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step Two consists of stating the verifiable student 
performance objective which consists of six elements: the 
student, content, behavior, time interval, method of measure- 
ment, and desired proficiency level. 

Step Three is a statement which outlines the planned 
program, or process, by which the desired change in behavior 
is to be accomplished. 

Phase II, the implementation stage, consists of placing 
into effect the program as detailed in Phase I. Throughout the 
implementation of the program, feedback is obtained which 
may indicate that the verifiable student performance objective 
or the process as stated in Phase I may, of necessity, be 
revised. 

Phase ill consists of measuring the product as speci- 
fied in the verifiable student performance objective. 

Phase IV, recycling, reinitiates the process by returning 
to Phase I where four alternatives present themselves (see 
Figure 4). 

If the desired behavior as stated in the performance 
objective has been attained, recycling consists of stating a 
new performance objective; presumably with the content 
sequentially dependent upon that which has been previously 
stated. 

If, on the other hand, the conditions of the objective have 
not been met, one must then consider the validity of the 
objective. The teacher may wish to merely revise the process 
— that is methodology or facility. Perhaps as an example, 
instead of using a filmstrip projector, she may wish to use an 
overhead projector. On the other hand, the teacher may re- 
consider one or more of the variables stated in the objective. 
Perhaps the proficiency level is unnecessarily high, the time 
interval too short, the number of students as stated is too 
large, or the students were not ready for the stated content. 
It must be emphasized that no one other than the classroom 
teacher can make a valid decision with respect to the vari- 
ables of a verifiable student performance objective. 

Still another possibility might be the actual deferment of 
the objective, either to a later date in the current school term 
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MODEL FOR RECYCUNG 




or even to the next school year. The mechanical processs 
involved in recycling will be discussed as part of the monitor* 
ing procedures. 

The monitoring system initialed at John Davidson con* 
sists of four phases: 

First A monitoring sheet is completed by each 
teacher on a weekly basis. 

Secondly The team leader critiques the monitoring 
sheets and offers suggestions and/or assist* 
ance to the teachers. 

Thirdly The monitoring sheets are logged into the file 
by the team's para*professional clerk where 
they are readily accessible to the entire staff. 

Finally The monitoring system is subject to interim 
audits at which time each teacher is inter* 
viewed by the auditor. 

The Monitoring sheet (Rgure 5) which was devised by 
the staff is printed on legal size paper to ensure ample space. 
An effort was made to keep it as simple as possible, yet pro* 
vide accurate and complete data. The front of the sheet 
contains spaces for the identification of the sheet. Information 
such as the sheet number, school week, school month, grade, 
teacher, and date are recorded. Below this heading are lines 
for writing verifiable student performance objectives. Each 
teacher numbers his/her objectives sequentially from the 
beginning of the school term; therefore, the numerals on a 
sheet are unique — that is, they represent the objectives of a 
particular teacher. Where an objective is indicated by numeral 
only, it may be found written out in full on a prior monitoring 
sheet. 

Notice that the objectives written out contain six vari* 
ables: the student, content, time interval, behavior, method of 
measurement, and proficiency level. While other objectives, 
those stated on prior monitoring sheets, are denoted by 
numeral only. 

The space for comments was added after the start of the 
school term. Teachers felt that details such as the transfer of 
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Figure 5. 
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MONITORING SYSTEM 
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students or loss of time due to some special event should be 
recorded. 

The right side of the monitoring sheet also contains space 
for identification. Belova the heading is a space labeled 
"Planned Program.” Observe that the horizontal dimension 
contains instructional variables, while the vertical dimension 
contains institutional variables. 

The instructional dimension consists of the variables of 
organization, content, method, and facility. While cost is a 
variable of this dimension, it is not generally (at least at this 
time) considered at the classroom level. Organization is that 
variable which considers time and space with time being 
defined as the duration and sequence of a particular block of 
teaching. 

Space refers to vertical or horizontal structure used in 
establishing the procedures by which students progress 
through school. Both structures permit either homogeneous or 
heterogeneous grouping as well as the typical organizational 
panaceas, departmentalization and team*teaching, and the 
traditional self-contained classroom.To the classroom teacher 
who is writing verifiable student performance objectives, it is 
apparent therefore that the type of organization is usually 
predetermined. 

For the purpose of an ongoing monitoring system, it is 
sufficient to state under the heading of organization the type 
of classroom, structure, the time to be spent on each objec- 
tive, and to indicate whether other personnel are to be 
involved. 

The variable referred to as content is probably the least 
troublesome since it is merely a statement of what is to be 
taught and has been stated in the verifiable student perform- 
ance objective. Briefly noted on the monitoring sheet, the 
content can then be related to the other elements of the 
instructional dimension. Some care must be exercised to state 
actual content rather than facility. An example may aid in 
clarification of this statement. If a verifiable student perform- 
ance objective involves the use of the glossary, this then is 
considered content; while the book in which the glossary Is 
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found is listed as a facility. Similarly, for example, the initial 
consonants "p, t, b, d” may be considered content, while the 
textbook or kit in which they are studied is listed as a facility. 

Slightly more involved is the variable labeled method. 
Here one considers teaching activities, types of interactions, 
and learning theory. Teaching activities range from lecture to 
Individual study and (while it is unlikely that more than two or 
three of the many choices in this category would be employed 
for a single objective) the typical teacher will be working on 
several objectives within a given subject area and therefore 
may be involved with numerous teaching activities which 
reflect his or her teaching style. 

Types of interaction consist of teacher-student, student- 
student, media-student and, in some cases, teacher-teacher. 
The latter is generally considered in team-teaching or where 
a student or group of students becomes involved with a spe- 
cialist or in a departmentalized situation. It is to be noted that 
the category 'Teacher-Student” is used whether the class, a 
group, or an individual student Is receiving help from the 
teacher. 

The theories of learning are many and diverse; therefore, 
it would seem appropriate to suggest that we consider our 
students as individuals with a wide range of capacity and 
motivation and to plan each objective so that it is meaningful 
and relevant to every student. Certainly each teacher must 
consider his/her individual philosophy in the writing of verifi- 
able student performance objectives. 

Facilities refers to the special space, materials and 
equipment, and expendables required for the expected 
change in behavior. For the self-contained classroom this 
usually consists of various audio-visual equipment, reading 
laboratories, workbooks, ditto master, textbooks, and even the 
chalkboard. As discussed previously, some care must be 
exercised to correctly identify facilities as such and not as 
content. 

The institutional dimension classifies people. For the 
purpose of monitoring, consideration is given only to those 
people directly involved in the process of bringing about the 
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desired change in the behavior of students. These may consist 
of the teacher, administrator, educational specialist, and the 
family. 

The planned program being used in this example depicts 
a self^ontained classrooni, a five-day week, and specifies the 
number of minutes per day for each objective. The content is 
the same as described by the objectives on the left side of the 
monitoring sheet. Notice the several types of interaction and 
teaching methods as well as the numerous facilities. Observe 
also that Matt is to go to speech two days, thus involving an 
educational specialist. Group II has homework which involves 
the family. 

At the conclusion of the week, Mrs. Smith has indicated 
that the planned program was carried out except that she 
decided to use a filmstrip projector and filmstrips relating to 
the initial consonants as indicated on the actual program. 

Below the actual program are spaces to indicate the 
status of the verifiable student performance objectives. Mrs. 
Smith has indicated that objective number 16 has been com- 
pleted and that objectives 19, 20, and 21 are not completed. 
Objectives 19, 20, and 21 will be identified on subsequent 
monitoring sheets by numeral only. During the time interval 
of the objectives, Mrs. Smith will be able to revise her program 
on a weekly basis if the need arises. 

Objective number 14 has been revised and has been 
designated as objective number 14 A. This notation clearly 
indicates that an objective has been recycled and that one or 
more of the six variables has been revised. 

An important component of any program, the intent of 
which is to satisfy the needs of accountability, is an evaluation 
design. While the only criteria specified in the district- 
mandated objective consisted of a five percentile improve- 
ment in total reading scores as determined by comparing the 
median score at each grade level on the State-mandated 
tests, the Davidson staff was adverse to the concept of com- 
paring two entirely different groups of students. That is to say, 
for example, this year’s second grade with iast year’s second 
grade. While the Davidson program employs a rather complex 
evaluation design, the basic components are PRE- and POST* 
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testing second end third grade students. Since no acceptable 
pre-test was available for first grade students, there was no 
choice but to compare this year’s first grade median scores 
with last year's. It should be noted that all testing was con- 
ducted in accordance with rigid procedures which included a 
parent-proctor in each room. All tests are scored by the team’s 
para-professional clerk and the scoring is then validated by 
the Team Leader. 

A further consideration has been given to student atti- 
tudes. An Attitude Opinionnaire was constructed, validated, 
and administered under very rigid conditions. Similarly, a 
parent attitude instrument has been constructed and admin- 
istered. 

The John Davidson Primary Reading Program represents 
an attempt by a staff to face up to societal demands for 
accountability and to paraphrase a statement from out of 
the past: 

"WE HAVE MET THE DRAGON AND HE IS OURS’’ 

PART III — Summary and Evaluation 

IN SUMMARY, the principle consequence of the Vallejo 
School District’s adoption of a program in Performance-Based 
Instruction was to re-position the responsibility for decision- 
making and reading achievement at the school level. As an 
extension of this concept, the Davidson Program evolved by 
active participation of several primary staff members who 
shared the responsibility for program design, selection of 
staff replacements, job assignments, direction in in-service 
education, and for many major operational decisions. 

At Davidson School, the so-called DECISION DOERS 
came from behind their classroom doors and became 
DECISION-MAKERS. As a consequence, it is worth noting 
that the final evaluation of this program will contain (1) an 
analysis of reading gains by individual class, (2) an analysis 
of student attitudinal changes by class, and (3) an analysis 
of the correlation of reading achievement and attitude by 
class. Additionally, there will also be an analysis of a parent 
attitudinal summary be grade. 
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IN CONCLUSION 

It should be stressed that any change is dependent to a 
large degree upon the willingness and ability of people to 
change their ways of doing things. Certainly, you will recall 
the allusion to the many hours of staff deliberation that 
preceded concensus. 

Further, even today, there is still measurable evidence of 
resistance and anxiety on the part of a few. However, in the 
final analysis, it may be truthfully reported that the majority 
of the teachers involved at John Davidson School have 
accepted the challenge of accountability and are committed 
to do their part in the first order of business — that is account- 
ability to their students and parents. 



COLLEGE LEVEL, WEBER STATE COLLEGE 
WILKIT PROGRAM 

DR. BLAINE P. PARKINSON, DIRECTOR 
IPT PROGRAM, COLLEGE OF EDUCATION 
WEBER STATE COLLEGE 
OGDEN, UTAH 

In the Fall of 1970 at Weber State College in Ogden, 
Utah, we initiated what we call an individualized performance- 
based teacher education program (IPT). The traditional 
lecture has disappeared from the professional education 
program. Teaching textbooks and testing footnotes have dis- 
appeared. Teachers are no longer in the business of enter- 
taining students. The grading game is no longer played. The 
threat of failure no longer hangs over the heads of students. 
If a test is failed, the student merely recycles. The quality of 
the student-teacher contact is greatly improving and it is no 
longer irreverent to call a faculty member by his first name, 
even for the Dean. 

In spite of this radical change, none of our buildings have 
been burned down nor have we had to defend ourselves from 
letters to the editor. However, we have had problems but 
mainly they are mechanical and can be easily resolved. 





52 



As we approached the development of our program, we 
knew of no teacher education program that was validated. 
No teacher education curriculum had been demonstrated to 
be any better than any other teacher education curriculum. 
We did not have the money to do a task analysis and build a 
curriculum from that vantage point. We were not certain of 
the best way for us to go; we did not want to start from the 
vantage point of the learner and then work back to teacher 
behavior. We simply began with ou r present teacher education 
program with its focus on teacher behavior. We changed the 
delivery system into an individualized performance-based 
format, one that had self-correcting potential. We developed 
self-instructional units which we called WILKITS, Weber 
Individualized Learning Kits. 

The elements of a WILKIT are as follows: 

1. The title is usually a single concept or skill. 

2. The scope suggests to the students what will be 
studied. 

3. Many of the WILKITS have a brief introduction which 
is a special message the faculty wants to give the 
student. 

4. The pre-test is used to diagnose the student for proper 
placement in the WILKIT. 

5. Behavioral objectives (which EPIC helped us develop) 
indicate what the student should be able to do when 
he has completed the WILKIT. 

6. The learning experiences are those activities which we 
think will best help the student to achieve the 
behavioral objectives (i.e., readings, seminars, public 
school experiences, films, tapes, peer teaching, micro- 
teaching, etc.). The learning experiences are optional. 

7. Each WILKIT has a self-test in order that the student 
can monitor his own progress. 

8. The final test is designed to measure the behavioral 
objective, not the learning experience. If a student 
needs other learning experiences, he comes to the 
faculty or designs them himself. For a final test, we are 
using oral interviews, objective and essay tests, and 
performance checkout for demonstrations of com- 
petence or a combination of these. 
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The average time to complete a WILKIT is five to thirty 
hours, and the average credit is one-half to one credit hour 
per WILKIT. Registration is by blocks of WILKITS. Blocks vary 
in credit from one to fourteen quarter hours. 

In the Secondary Education Program, there are twenty- 
five WILKITS. In the Elementary Education Program, there 
are fifty WILKITS. Students register at the regular registration 
time, but they can complete them any time they want to. If 
they want to start a block of WILKITS before registration, they 
may do so. They may have them completely done by the time 
registration rolls around. One year is allowed to complete 
any one block of credit. 

We have developed a center which we call "Operations.” 
In this center, students check out WILKITS, sign up for sem- 
inars, make appointments to see faculty members, arrange for 
peer teaching, or take a test at any time. We are trying to 
develop a very informal, pleasant area where students feel 
comfortable because this becomes a major contact with the 
Education Division now that no lectures exist. 

With an individualized program, any one activity such as 
film, seminar, audio tape, video tape, peer teaching, or micro- 
teaching must be rotated regularly so that a student will not 
have to wait very long to get the activity. This puts an entirely 
different instructional load on the faculty. Rather than assign- 
ing load in terms of hours of credit or in terms of number of 
WILKITS supervised, we assign load on the basis of seminars, 
performance checkouts, and individual conferences. This 
year v/e have had an unrestricted schedule. A student could 
schedule any activity at &ny time of the day. He goes to 
Operations and requests that an activity be scheduled. 
Faculty have given certain times throughout the day which 
Operations can schedule them for. If a student and Operations 
.set up a Seminar at a particular time, then a list is posted and 
any other students who wish to attend may sign up. We prefer 
seminars with approximately ten students. The open schedule 
put too heavy a demand on faculty and too many seminars 
were held with two or three students. We did not request 
students to leave time in their schedules for scheduling 
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activities relating to WILKITS; thus, many students found it 
difficult to find the time for these activities. 

We now require students to schedule any block In one 
of three alternative hours, such as 8:00 a.m., 11:00 a.m., or 
1:00 p.m. The faculty Is available at those hours for small 
groups and Individual activities, and at other hours for 
individual activities. 

We have had two concerns with respect to our tests in 
WILKITS. First, we are trying to measure the behavioral objec- 
tives and not the learning experiences. This move toward 
criterion-referenced tests has not been easy nor have we 
reached the standard which we wish to achieve. Secondly, 
with an individualized program, we must be able to test any 
student at any time which is a problem in terms of security 
with test materials. We hired a group of instructional tech- 
nology people to work with our faculty in doing a thorough 
revision of our tests developing several alternate forms and 
In validating them. 

We want to evaluate the students’ ability to apply the 
skills which are learned in the WILKITS In student teaching or 
in the real teaching situation. To do this, we are developing 
an observation instrument built on the behavioral objectives 
of the IPT Program. EPIC assisted in the development of this 
observation instrument. It will be somewhat like Roberson’s* 
and Flander’s* observation instruments, but will be unique 
not only in content but in format. 

We cannot evaluate all of the skills of a teacher, nor are 
we certain what these skills are. But if we cannot teach and 
evaluate the performance of some teaching competencies, 
then we had better close shop. To facilitate evaluation and 
offer a more realistic opportunity to apply, we need a new 
type of student teaching situation. We are trying to move from 
theory to practice, under controlled conditions, to a realistic 
application experience. Most of our WILKITS have some 
public school exposure. For instance, we have Elementary 
Education students in the schools about sixty-six hours prior 



*DovoloptnQ Obsorvation Skills, Educational InnovotofC Pross, Tucson, Arizona, 1971, 
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to student teaching, besides this they’re doing a good deal of 
peer teaching and microteaching. In the past students have 
always said, teacher education is too theoretical. Now they 
are saying, I see where it works. We are trying to teach them 
the application of theory and give them practice under a 
controlled environment then hold them accountable for appli- 
cation out in the Teaching Practicum Centers. In the past, 
student teaching has been too unreal. Often a student teacher 
has gone into the school, observed, then mimicked the 
teacher, sometimes he has had an opportunity to try his ideas 
and sometimes not. As an alternative to this, we have tried 
teaching internships where a student takes over a classroom 
at one-half salary. In this setting he does not have time to 
really see what is happening to him, to get feedback and 
really evaluate himself. 

As an alternative approach we are developing teaching 
practicum centers. Each center will be led by a clinical 
teacher, who knows what our concepts are and what skills 
are being taught in the WILKITS. This teacher and another 
good professional teacher could take up to three student 
teachers at any one time. When those three student teachers 
have completed their training, three other student teachers 
could be phased in to take their places. The center might 
handle from 75 to 1 25 public school students. There would be 
adult supervision to allow much flexibility in the unit. The 
clinical teacher would be the team leader and would be 
expected to plan regularly with the group. The typical co- 
operating teacher does not know the college program and 
does not offer adequate supervision. The college supervisor 
often dissipates a great deal of time travelling the circuit to 
check on a few student teachers and very often isn’t able to 
make adequate observations of the student teacher. 

In the new approach, we maximize planning, maximize 
supervision, maximize feedback and maximize opportunity to 
teach. We can utilize the college supervisor as a consultant 
in the area of his specialty. He could work right with the team 
for a period of time. 

Now, as to why we have discontinued the grading game. 
We agree with many that A, B, C grades are not serving educa- 
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tion well and have probably done a lot more harm than good. 
The most you can say about an A is that it is better than a B 
in some undefined group. An A in one group may be lower 
than a D in another group, we just do not know what it means. 
We have been in a norm reference system where we compare 
students with each other. Now that is essential and adequate 
in some types of instructional situations. To compare students 
with each other may be the only base that makes sense. The 
whole game of testing in a norm reference system is to spread 
people out. Most academic vice presidents send letters to 
faculty instructing them not to give everybody A’s or to not to 
give everybody B’s. They say "You’ve got to spread them out.” 
We expect variation in achievement. We force variation in 
achievement. In a criU rion-referenced system which we hope 
we are moving toward, we set a clearly-specified goal. If one 
meets the goal and wants to go on, let’s set a nevy goal, let’s 
not give him a lot of busy work tr give him an A. 

In the traditional system, there are certain things we hold 
constant. We hold instruction constant, we hold assignments 
constant, we hold time constant, but we vary achievement. It 
seems to us that it makes more sense to vary instruction, vary 
the assignments, vary time but hold achievement constant. 
How else can we have quality control and ensure excellence? 
Let’s set the standard, hold to it, then manipulate the variables 
to achieve the standard. If you want kids to turn out different, 
you should treat them all the same. If you want them to turn 
out all the same, you have got to treat them differently. 

One of the most successful components in our program 
has been the Interaction Laboratory. We knew there was 
some value in group work. We knew our teachers were not 
strong in human relations skills. Your children and my children 
all continually come home and tell of experiences where 
teachers turn them off because of poor human relation skills. 
We searched for a long time for a program that might get at 
human relations training and finally contracted with Thiokol 
Chemical Company of Ogden, Utah, to develop what we call 
an Interaction Laboratory. 

It Isa group experience with twenty-six 90-minute exer- 
cises. It is experiential but structured to teach essential 
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human relation skills. It met for one hour and a half a day for 
six weeks or for three hours a day for three weeks. Groups 
consist of about fifteen students with a trainer and a co- 
trainer. The trainer is a faculty member. A co-trainer is a 
senior student who has been through the IPT Program. 

Many of the stimulus experiences are drawn from teach- 
ing. The format Is as follows: A short activity Is participated In 
like many group programs have. The bias exercise is a very 
interesting one. We show the group a film of an Interview with 
a third grade girl. We give half of the group j; 'it by telling 
them that the girl is the daughter of a cocktail waitress who 
has moved several times, a rather poor student in school, etc. 
For the other group, we tell them she Is the daughter of a local 
doctor, they are fine people and that the girl is a good little 
student. They are asked to observe the film and make some 
judgments on the basis of the film. They are asked if they 
think she will graduate from high school or if she will relate 
well with adults. When we tally the results, we find half the 
group answering the questions one way and the other half 
answering the questions another way. We ask them, simply, 
what happened here. At the beginning of the lab they learn 
that we won’t preach, it is up to them to find the answers. We 
just ask a few questions. For instance, we say "what hap- 
pened.” They say, “we were trapped in our bias." Then, we 
ask, "what does this mean for a teacher?” In the group discus- 
sion that follows, students explore very thoroughly what that 
means for a teacher and come out of the session with some 
new insights. 

The lab is constructed on a variety of such exercises, 
grouped in four basic areas: communications, group building, 
interpersonal, and professional skills. Students say it is the 
most significant experience they have had in college. It has 
established a new student-faculty relationship. I have always 
been called Dr. Parkinson. Even by students I have really 
helped, but now the tone of voice in which students call me 
Blaine rather than Dr. Parkinson tells me they are different. 
There is a trust and an openness here that I have never had 
in twenty years in teaching. 

Thiokol Chemical Company unfortunately kept the copy- 
right for the laboratory. They are now marketing it. It has 
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turned out to be equally effective for in-service teachers, para- 
professionals, and job corps teachers and I am certain would 
be useful with many other populations. 

We have not been in business long enough to evaluate 
yet. We spent most of the first year asking students how they 
liked the program and what was wrong with the WILKITS. 
We have been amazed at how well we can respond to their 
critique and modify the particular objective or learning experi- 
ence and students were amazed that we would do so. Student 
critique changes traditional courses very little. I think the 
difference is that now if a student says, "I don’t like the 
WILKIT.” We say “What Behavioral Objective or what learning 
experience are you talking about?’’ Then we can pinpoint the 
trouble. 

Student responses are as follows: “Last year I took two 
education courses. I figured I’d sit there and do what I had to 
do and get a C. So I did, I got a C and learned zero.’’ “You 
know this year I’ve had to learn something to pass the WILKITS 
and do you know what, I’ve really enjoyed it.” “Behavioral 
objectives are great.” “I know where I’m going and I know 
when I’ve achieved it.” “For the first time in college I’m learn- 
ing to take responsibility for my education.” “You know I can’t 
stand to sit through a dry lecture any more.” “I’ve used the 
skills and materials in my Student Teaching.” 

We have mainly tried to sophisticate our procedures, 
learning experiences, and tests this first year. We have had 
enough of the above type comments by students that we feel 
very positive about the new program. We know we are in a 
completely different ball game and that the students are 
seeing themselves and teacher education very differently. 
The big task of evaluation lies ahead. We must demonstrate 
that they can apply the skills in the classroom. Then we must 
look at how well the people they teach learn. 

We had a $200,000 grant with Carnegie Corporation of 
New York to develop the IPT Program. We are attempting to 
find money now for evaluation, research, and revision of the 
program. 

We think we have a program that will be self-correcting 
that we can continually modify. We have got our instructors 
out of the notion that a particular course is theirs and it must 
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be a certain way. That is the first thing we tried to break down. 
Nobody has a ciaim on any part of the curricuium. if we do not 
iike a behaviorai objective of a WiLKIT, we throw it out. 

There is no question in my mind that the performance- 
based individuaiized approach has taken Teacher Education 
out of the horse and buggy stage and into the Modei T stage. 
The Modei T stage may not be much better now than the oid 
system, but as we combine our efforts, we wiii soon be into 
the Cadiiiac and jet stage and may deveiop a true technoiogy 
of teaching. 
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